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Abstract 

Background: As a public safety regulator, the Technical Standards and Safety Authority (TSSA) of Ontario, Canada 
predicts and measures the burden of injuries and fatalities as its primary means of characterizing the state of public 
safety and for decision-making purposes through the use of a simulation model. The paper proposes a simulation- 
based predictive model and the use of disability-adjusted life years (DALYs) as a population health metric for the 
purposes of reporting, benchmarking, public safety decision-making, and organizational goal setting. The proposed 
approach could be viewed as advancement in the application of traditional population health metrics, used primarily 
for public health policy decisions, for the measurement and prediction of safety risks across a wide variety of engineering 
technologies to which the general public is exposed. 

Results: The proposed model is generic and applicable to a wide range of devices and technologies that are typically 
used by the general public. As an example, a measure of predicted risk that could result from the use of and exposure 
to elevating devices in the province of Ontario is presented in terms of the DALY metric. The predictions are further 
categorized in terms of the causal attribution of the risks for the purposes of identifying and focusing decision-making 
efforts. The results are also presented by taking into consideration factors such as near-misses or precursor events as 
termed in certain industries. 

Conclusions: The ability to predict potential health impacts has three significant advantages for a public safety 
regulator - external reporting, decision-making to ensure public safety, and organizational benchmarking. The application 
of the well-known Monte Carlo simulation has been proposed to predict the health impacts expressed in terms of 
DALYs. The practicality of the proposed ideas has been demonstrated through the application of the prediction model 
to characterizing and managing risks associated with elevating devices in the province of Ontario, Canada. 



Background 

Information and data on incidents and near-misses are 
analogous to the visible tip of the iceberg that enables 
the regulator to be reactive to unfortunate occurrences. 
There may also be valuable data such as noncompliances 
obtained typically through regulatory inspection programs 
that identif)^ underlying or nascent failures in the regu- 
lated systems that support proactive decision-making. 
Additionally, there could be unforeseen scenarios that are 
best anticipated through a thorough predictive risk assess- 
ment. Noncompliances, near-misses, and incidents form 
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valuable predictive inputs to prevent fatalities and injuries. 
These represent a triangle: the base provides opportunities 
for improving public safety, and the severity increases as 
we reach the top in Figure 1. The inherent and unobserv- 
able states at the base of the triangle represent faults in 
the regulated engineering system that are not observed 
until identified during an inspection or an occurrence is 
reported. 

A judicious choice of a population health metric for 
the purposes of quantifying risk that is predictive in na- 
ture and a risk-acceptability criterion as a benchmarking 
standard for organizational goal setting are essential. 
Hence, the objective of this paper is to propose a tech- 
nique for measuring and quantifying risk in the context 
of regulated devices and explore the application of 
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disability-adjusted life years (DALYs) as an appropriate 
metric to predict future health burden. 

The TSSA is a not-for-profit self-funded organization 
that administers and enforces public safety laws in vari- 
ous sectors under Ontario's Technical Standards and 
Safety Act. The regulated sectors include elevating and 
amusement devices, fuels, boilers and pressure vessels, 
operating engineers, ski lifts, and upholstered and 
stuffed articles. There are a large number of devices, 
technologies, and products (herewith known as technical 
systems and products) with which the public interacts 
on a daily basis across the province of Ontario. The state 
of compliance of these technical systems and products 
to the specific codes and standards is ensured through a 
collective application of regulatory instruments that span 
the life cycle of these systems. Active information on the 
state of compliance is monitored and collected primarily 
through periodic inspections of the technical systems 
and sometimes, through random ad hoc inspections. If 
noncompliances are identified, they are enforced and 
logged in to an information system. Despite the efforts, 
there are sometimes incidents and near misses, some of 
which lead to health impacts in the general public. For 
the purposes of distinction, an incident may result in a 
consequence such as an injury or property damage, 
while a near miss results in an elevated exposure that 
could potentially lead to such consequences. Incidents 
and near misses are collectively known as occurrences at 
the TSSA. 

The DALY has been explored by various public safety 
authorities in Canada and around the world including 
the Australian Institute of Health and Welfare [1], Na- 
tional Center for Infectious Diseases, USA [2], National 
Institute for Public Health and the Environment, 
Netherlands [3], Public Health Agency of Canada [4], 
Statistics Canada [5], and Health Canada [6]. The TSSA 
in particular has been using DALYs as a reporting metric 
since 2010 [7]. A DALY is the loss of a year in an indi- 
viduals life who would have otherwise led a healthy life 
[8]. The intent is to utilize DALYs in a decision-making 
setting as a single dimensional metric [9] resulting from 



aggregating morbidity and mortality outcomes. TSSA 
has already used this approach to risk measurement in 
prioritizing and managing several known safety issues 
associated with elevating devices [10]. 

The concept of using the DALY metric in the future 
domain has been explored by studies related to the 
Global Burden of Disease (GBD) [11,12]. Disease- 
specific projection studies elsewhere have taken inspir- 
ation from the GBD literature [13]. These studies have 
extensively applied multiple linear regression on large 
amounts of global individual mortality data available to 
the World Health Organization (WHO). The objective 
of these studies is to estimate the mortality rate with de- 
pendency on various covariates such as smoking, level of 
education, age. etc. The future burden of disease is then 
reported in terms of DALYs. The TSSA has unique chal- 
lenges in the sense that it does not collect in-depth 
device-specific intrinsic factors, which could potentially 
be causes of unfavorable human health impacts in the 
event of occurrences. Some of the factors are age-related 
susceptibility to degradation mechanisms that accumu- 
late over time, making a device high risk in nature. It is 
quite common in the engineering failure analysis litera- 
ture to use this information to predict the state of a de- 
vice in its future course of usage. As a result of the 
missing covariate information, the well-explored concept 
of regression analysis is not applicable for prediction 
purposes at the TSSA. 

However, there is a wealth of credible historic occur- 
rence data along with actual health impacts collected 
over a period of several years. Hence, the present paper 
effectively utilizes the available occurrence data in order 
to predict potential health impacts by implementing a 
second-order Monte Carlo simulation. The predicted 
health impacts in terms of DALYs are reported as a 
range of percentiles so that readers can appreciate the 
extremities along with the average estimates. 

Methods 

Model overview 

A flowchart of the first-order Monte Carlo simulation 
for predicting health impacts in terms of DALYs is pre- 
sented in Figure 2. This level of simulation captures vari- 
ability in number of victims, injuries, injury types, and 
age of the victims. However, at this stage uncertainty in 
the occurrence rate \ is ignored. The end result of the 
first-order simulation is a DALY distribution classified 
by causal categories. 

Keeping in view the importance of separating variabil- 
ity and uncertainty through a second-order simulation 
[14], uncertainty in X could be exclusively dealt with in a 
second layer of simulation through the reporting bias. 
This separation increases the model complexity and be- 
comes computationally prohibitive [15]. 
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Gather Occurrence Data: Incidents, Near-Misses 



Assume every incident and near-nniss without injury has a 
potential for being a future incident with health innpacts 



Deternnine average Occurrence rate 





Sample the number of Occurences 










c 
o 


Predict number of injured victims 




Age Distribution 














m 


T 




D 










E 


Predict number and type of Injuries 




Injury Distribution 




</) 










T 




L 


Predict health impacts 




DALY short- and long-term weights 





T 



Predicted DALY 

Predicted DALY by Causal Category 
(per million per year) 



Figure 2 First-Order Monte Carlo simulation (fixed A) for DALY prediction. 



Establishing an occurrence rate 

The TSSA database has an account of each reported oc- 
currence along with factors such as the date of occur- 
rence, whether the occurrence is an incident or a near 
miss, the number of injured victims, and number and 
type of injuries sustained by each victim. Actual or ob- 
served DALYs do not account for near misses and inci- 
dents without injury. Hence, an occurrence rate based 
solely on past occurrences with health impacts does not 
truly reflect what would potentially happen if some of 
the near misses manifest themselves as occurrences with 
health impacts in the future. But then the uncertainty 
lies in how many of the nonhealth impact occurrences 
turn in to incidents with injuries. We assume two ap- 
proaches to account for the benefit uncertainty: (1) as- 
sume that every such occurrence carries an injury 
potential in the future (r = 1) or (2) assume a near miss 
to incident ratio based on the safety pyramid rule [16] 
from the chemical processing industry (r > 1). The sec- 
ond assumption has the motivation in the belief that 
some of the near misses may never have negative health 
impacts purely by design. For example, activation of 
relief valves or rupture discs on pressure vessels is de- 
signed to manage overpressure scenarios. These activa- 
tions may be reported as near-miss pressure boundary 
failures. 



Then, the occurrence rate \ is established in terms of 
the number of incidents rii, number of near misses 
and the observed duration Tas: 

^= (1) 

This expression determines the number of occurrences 
for an iteration of the simulation as a sample from Pois- 
son distribution with \ as the mean occurrence rate. It is 
assumed that the number of occurrences has been con- 
stant over the years. However, it is of future interest to 
move toward a time-dependent occurrence rate \(t) so 
that the number of occurrences in a future year is pre- 
dictive rather than assuming a past average. A nonho- 
mogeneous Poisson process with an underlying power 
law model is an alternative. 

Number of injured victims 

The number of injured victims in the event of an occur- 
rence is a random variable. This number is dependent on 
the technical system or product under consideration. For 
example, in the context of elevating devices, the number 
could be anywhere between 0 and the maximum observed 
in the history or the mean number M of passengers a typ- 
ical device could affect. It is assumed that the number of 
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injured victims follows a discrete uniform distribution 
with parameters a = 0 and b=M, Historically, there have 
not been more than three injured victims in a single oc- 
currence associated with elevating devices. 

Injury type distribution 

An inspector is usually dispatched in the event of an oc- 
currence to gather circumstantial evidence. The victims, 
if any, are interviewed for their age and types of injuries 
sustained and regarding the situation in general. The 
collected injury information is fed in to TSSAs Incident 
Management Information System (IMIS) tool. The his- 
torically observed injury type distribution is used for 
simulation purposes. For example. Figure 3 shows the 
injury type distribution observed for elevating devices 
between 2009 and 2011. Though there have been a few 
fatalities and permanent injuries affecting life expectancy 
of victims in the usage of elevating devices, the majority 
of the injuries have been high-incidence, low- disability 
conditions. A fatal injury equates to death and hence all 
other injury types are discarded in the event of a death. 
There could be a few injury types that may not have 
been observed in the past. In such cases, all the unob- 
served types are pooled so that the combined frequency 
sums to one. This ensures that the unobserved types are 
picked up during simulation but with a lesser chance. 
The TSSA inspectors choose each of the applicable sus- 
tained injury types exactly once during data collection 
on the field. However, the simulation process samples 
the types with replacement in order to reduce computa- 
tional burden. 



Determining the DALY 

The expression for calculation of DALYs adopted from 
the AIHW is given as [1]: 

DALY = Short-term Weight * Short-term Duration 
+ Long-term Weight * (Fraction Long-term 
* Long-term Duration) 

(2) 

An injury sustained can have either or both of short- 
term and long-term health impacts. The weights chosen 
by AIHW [17] were in turn adapted from the GBD stud- 
ies at WHO [18]. The long-term duration is the average 
life expectancy of the victim at the time of the occur- 
rence. For the purposes of simulation, the life expect- 
ancy is averaged across gender. Since a long-term 
disability may not last for the rest of the life, the dur- 
ation is weighed by the fraction long-term. The assumed 
weights and durations are listed in Table 1. 

A DALY estimate from Equation (2) for fatality of an 
individual is determined by setting the short-term pa- 
rameters (disability weight and duration) to zero, the 
long-term parameters (disability weight and fraction) to 
one, and long-term duration as the remaining life ex- 
pectancy of the deceased. The result is a DALY value 
which is equal to the victims expected life expectancy at 
the age of death. In a general setting, if the remaining life 
is set to median life expectancy in Ontario (44.4 years; 
subject to change), the resulting DALY associated with 
mortality could be used as a benchmark and guiding 
principle for decision-makers. 
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Figure 3 Injury type relative frequency distribution: elevating devices, 2009-2011. 
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Table 1 Disability weights related to TSSA specific injury types 



Burden of disease injury type 


TSSA injury type 


Short-term 
weight 


Short-term 
duration 


Fraction 
long-term 


Long-term 
weight 


Intracranial injuries [17] 


Concussion 


0.359 


0.0671 


0.05 


0.35 


Poisoning [17] 


Poisoning 


0.611 


0.0082 


0 


0 


Injured spinal cord [19] 


Spinal injury 


0.725 


0 


1 


0.725 


Sprains [17] 


Sprained or twisted 


0.064 


0.0384 


0 


0 


Internal injuries [17] 


Other internal injury 


0.208 


0.0425 


0 


0 


Upper respiratory infections - pharyngitis [19] 


Respiratory infection 


0.07 


0.02 


0 


0 


Rheumatic heart disease and heart failure [20] 


Heart attack 


0.323 


0.1 


0.2 


0.353 




Aches or pains 


0.02 


0.02 


0 


0 




Electric shock minor 


0.04 


0.02 


0 


0 




Electric shock severe 


0.2 


0.1 


0.1 


0.15 




External bruise 


0.04 


0.02 


0 


0 




Swelling 


0.04 


0.02 


0 


0 




Whiplash 


0.04 


0.02 


0.05 


0.04 




Fatal injury 


0 


0 


1 


1 




No Injury 


0 


0 


0 


0 



Adaptation of AIHW disability weights to specific TSSA 
injuries 

The injuries observed from occurrences resulting from 
TSSA-regulated technologies (Table 1) do not corres- 
pond exactly with the injury types in the AIHW tabula- 
tion [17]. Where the TSSA-defined category did not 
evidently correspond to one from the Victorian study, 
the most appropriate proxy was considered, e.g., intra- 
cranial injuries for concussion. If a certain TSSA injury 
type was associated with subcategories in other studies, 
the most conservative one was chosen. As an example, 
for respiratory infections, the disability weight associated 
with pharyngitis was chosen from the GBD 2004 update 
[19]. Some injury types such as heart attacks have been 
reported to the field inspectors in the past. These are 



possible health outcomes from several scenarios such as 
entrapments, electrical shocks, falls, etc. during elevator 
rides and hence have been included in the list of injury 
types. Rheumatic heart disease was chosen for short-term 
disability and heart failure was chosen as the long-term 
one [20]. In case of fatality, disability weight and fraction 
of long-term duration are both assigned a one so that the 
DALYs are set to the life expectancy of the victim. All pa- 
rameters are set to zero for "no injury" in order to keep 
track of the number of occurrences where there was no 
injury. TSSA regulates various sectors, the injury types 
across which range from minor injuries to permanent 
ones, including mortality. The current choice of injury 
types is subject to revision based on shortcomings iden- 
tified [21] in relation to differentiating high-incidence 
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Figure 4 Relative frequency distribution for causal categories. 
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low-severity injuries from low-incidence high-severity in- 
juries and also based on the improved methodology to 
measure disability weights [22]. As a proof of concept, 
high-incidence injury types (e.g., aches and pains) that do 
not correspond to the GBD study have been assigned a 
very low disability weight (< 0.02) and an approximate 
duration of 1 week (< 0.02). These assumptions are 
needed until a more robust theory is articulated on deter- 
mining DALYs for injuries rather than for diseases. The 
authors are of the opinion that public safety decision- 
making in sectors dominated by injuries is better informed 
with certain assumptions for disability weights, although 
some of them cannot be directly mapped to the burden of 
disease studies. This scheme is more informative than 
making decisions solely based on actual number of ob- 
served infrequent fatalities. 

Occurrence cause classification 

It is believed that incident and near miss occurrences 
are the ultimate realization of safety risks and can result 
from any one of three causal safety impact measurement 
(SIM) categories: 

• SIM 1 - causal factors related to potential 
inadequacies in available regulatory controls 

• SIM2 - causal factors related to noncompliance 
with current regulatory controls 

• SIM 3 - external causal factors not within the 
purview of current regulatory controls (e.g., 
behavioral factors, weather conditions, etc.) 

Occurrences that do not have an established root 
cause after inspection are contained in a fourth category, 
root cause not established (RCNE). 

For each simulated occurrence, one of the causal cat- 
egories is sampled based on a relative frequency distri- 
bution of past observations. For example. Figure 4 



shows the distribution for the case of elevating devices 
observed over the past 4.6 years. 

There are sectors that TSSA regulates in which high- 
incidence low-disability conditions are quite common. 
In such circumstances, TSSA continues to focus on fre- 
quency of system-induced failures and causal reasoning 
alongside the risk knowledge driven by the DALY metric 
in order to undertake strategic public safety decisions. 

Results 

Application to elevating devices 

The proposed model was implemented to understand 
the risks associated with the use of and exposure to ele- 
vating devices in the province of Ontario on a pilot 
basis. There are over 48,000 elevating devices including 
escalators in the province of Ontario with a susceptible - 
to-risk population base of about 13.4 million people. 
The assumption is that every Ontarian is equally ex- 
posed to the risk of an elevating device occurrence at 
some age in their lifetime. The reported occurrences 
were riyn = 1028 near misses and rii = 2638 incidents 
(2,500 with injuries and 138 without injuries). Experts at 
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Figure 7 Predicted DALYs witliout reporting bias. 



TSSA believe that nearly 10 occurrences go unreported 
for each reported occurrence. It was assumed that there 
could be anywhere between zero and six riders in a typ- 
ical elevator and that each injured victim could sustain 
up to four injuries. As a frame of reference, Figure 5 
shows the actual observed DALYs in the past T = 
4.6 years. The mean occurrence rate represented in 
terms of a million people per year is shown in Figure 6 
for the cases where = 0, r=10, and r=l. r=l has 
been used at TSSA for reporting purposes which conser- 
vatively assumes that every near miss has a potential to 
result in a future incident with injuries. r=10 does not 
have a strong basis to defend, however serves as a sensi- 
tivity analysis in case only one out of 10 near misses re- 
sults in incidents. The simulation was run to yield mean 
DALYs in the absence of reporting bias. The results are 
shown in Figure 7 categorized by the SIM classification. 
While the average of the prediction has been selected 
for reporting purposes, the simulated DALYs form an 
empirical distribution. A single case with near misses 
and reporting bias omitted is shown in Figure 8. 

Discussion 

In the context of elevating devices, there have been four 
fatalities observed in the past, and the predicted ex- 
pected health impact is about 22 DALYs per million 
people per year in the absence of reporting bias. DALYs 
for the cases without near misses and with r = 10 in 
Figure 7 are the same (0.5), signifying the possibility that 
though there are apparently many near misses happen- 
ing in the province of Ontario, only a few are being ac- 
tively reported to TSSA. Assuming a one-to-one ratio 



between incidents and near misses, the DALY estimate 
is 26.6. The predicted SIMS risk (1.9 DALYs) indicates 
that occurrences related to elevators in Ontario are 
dominated by user behavior and factors external to the 
regulatory system. A number of occurrences under 
RCNE were recategorized at the end of the year as user 
behavior-related. In order to reduce risk related to aging 
systems that do not conform to the current elevator 
codes, the elevator risk reduction group analyzed eleva- 
tors with single- and two-speed motion control and the 
associated risks to passengers when the elevators do not 
stop at the prescribed level. This analysis could lead to a 
replacement of older control systems in the near future. 
In order to tackle public safety risk due to noncompli- 
ance, TSSA issued a communication to elevating device 
owners reminding them of their legal responsibilities 




19.2 19.8 20.4 21.0 21.6 
Predicted DALY (per million per year) 



Figure 8 Predicted DALY distribution withiout near misses and 
reporting bias. 
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with regard to elevator maintenance and oversight of 
their maintenance contractors. The owners were advised 
of new requirements and the timeline in which they will 
become effective so they can better plan for improved 
compliance. Figure 8 shows the predicted health impacts 
as a distribution to illustrate the range of health impacts 
predicted in the elevating devices sector. 

The proposed model supports risk-informed decision- 
making and hence has a potential to influence policy im- 
plications at the organizational level. In particular, the 
model has significant importance for technical systems, 
such as those involving high pressure boilers and vessels, 
where there is possibility of a high-consequence, low- 
probability incident with health impacts to the popula- 
tion, even though most of such occurrences are rare 
events making it difficult for policymakers to take risk- 
informed safety initiatives. Secondly, the model could be 
explored for new technologies being brought in to the 
purview of the regulatory system for which there is no 
health impact data or where there is an effort to 
strengthen the data management system of an existing 
technology. While the proposed model strives to bring 
predictive power to risk assessments, traditional systems 
analysis of engineering devices based on failure frequen- 
cies continues to additionally inform recommendations 
for public safety decision- making. 

Conclusions 

The present paper explored the application of DALYs, 
traditionally used as a population health metric, for pur- 
poses of measuring and quantifying public safety risk in 
the context of engineering devices and technologies. In 
the absence of extensive covariate information, it was 
shown that using a Monte Carlo simulation, technology- 
induced health impacts could be predicted for the fu- 
ture. The simulations combine mortality and morbidity 
to yield future health impacts (or public safety risks) in 
terms of DALYs expected in a given year. The categoriza- 
tion of DALYs into causal factors allows for policy- setting 
and resource allocation. The proposed model was demon- 
strated in the context of characterizing and managing 
public safety risk for elevating devices in the province of 
Ontario, Canada. 
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